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Sir: 

REPLY BRIEF 

Appellant hereby submits this Reply brief under the provisions of 37 C.F.R. 
1.193(b) in response to the Examiner's Answer mailed March 4, 2010. 

The arguments set forth in the Appeal Brief dated December 7, 2009 are 
incorporated herein by reference, and Applicant will not repeat the same herein. The 
following arguments are presented in response to new arguments presented in the 
Examiner's Answer and to further clarify Applicant's previous positions. 



General Remarks 

Applicant submits that Madrane, the sole prior art reference used in the rejection 
of all pending claims, is used in a completely different way, and for a very different 
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purpose, from that of the invention disclosed in the instant application. The present 
invention is concerned with organizing and storing media objects (see claim 1). As 
discussed on page 1 lines 29-35 of the instant specification, a media object is a data item, 
or pointer to such a data item, related to a media file (text, image, sound file etc) 
and containing metadata elements which represent classification categories relevant to 
that media file. It may be considered as analogous to an index card in a catalogue - it 
represents, but is not part of, the file to which it relates. The individual metadata tags are 
the various classification marks indicative of the content of the file - for example whether 
a film clip is text, music or video; drama or documentary; who created it, who appears in 
it, etc (see for example Figure 4) 

It appears that the Examiner is interpreting the claims as if the media objects are 
elements within the content of a video stream, for example an image of a person, which is 
tagged so that it can be tracked from each frame to the next as it moves across the field of 
view. These are part of the file content - more like a bookmark than an index card. The 
specification taken as a whole does not support any such interpretation. 

Claim 1 (similarly for claim 15) requires an object to be "selectively moved by a 
user into a region of the display representing a selected set of metadata tags". Thus the 
tags are primarily associated with the region, not the object . They only become associated 
with the object when the object has been moved, by the user, into the relevant region. 
(The region is a tagged "bin": the user chooses which object goes into which bin). In 
contrast, in Madrane, the tag is permanently associated with the relevant object , 
and moves with the object either as a user manipulates an image containing the object, or 
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because the object is depicted in a film clip as moving across the field of view. The tag is 
not directly associated with a display region, but only with the object currently depicted 
in that region . 

The Examiner stated (see p. 3 of the Examiner's Answer) that the designer of the 
system can designate any particular area of an image to be associated with a tag (a url). 
But such an area would simply be another tagged object. Madrane does not suggest 
having the end-user moving other objects into and out of that tagged area (e.g., 
manipulating the content of a film clip so that a person is added, removed, or 
repositioned, relative to the background in a particular view), let alone doing so to alter 
the tags associated with that object. In Madrane, the tags identify the object, not its 
location. 

In Madrane, the tag is permanently associated with the object. In claims 1 and 15. 
the tag is associated with a specified part of the display area , and with any objects that the 
user has, for the time being, placed in that area. If the user removes an object from that 
area, the tag ceases to be associated with that object, but remains associated with the area, 
and with any other objects remaining there. 

In summary, the metadata applied in Medrane is applied during creation, not 
afterwards: by the creator of the data, not its end user: and in order to manipulate the 
content, rather than to classify it. 
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1. First Reply Argument 

With respect to the issue of whether claims 1 and 15 are anticipated under Section 
102(b) by Madrane (US 6,573,907), and more specifically, with respect to the issue of 
whether Madrane teaches displaying on the display various regions representing various 
values of metadata tags, the Examiner stated that Madrane teaches display regions (e.g., 
regions associated with the image of the person walking, or the image to be zoomed) 
representing values of the metadata tags, because the metadata associated with the tags 
attached to the images that are currently in the region direct how the images are to be 
displayed, see p. 10 of the Examiner's Answer. 

Applicant submits that in claims 1 and 15, the tag is associated with a specified 
part of the display area, and with any objects that the user has, for the time being, placed 
in that area. If the user removes an object from that area, the tag ceases to be associated 
with that object, but the tag remains associated with the area, and with any other objects 
remaining there. In contrast, in Madrane the tags are permanently associated with the 
objects, not with a specified part of the display area. 

2. Second Reply Argument 

With respect to the issue of whether claims 1 and 15 are anticipated under Section 
102(b) by Madrane (US 6,573,907), and more specifically, with respect to the issue of 
whether the center of the screen in Madrane (to which the object of interest may be 
placed by the designer) represents a selected set of metadata tags, the Examiner stated 
that when the designer designates an object as interesting and places it in the center of the 
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screen and then zooms on it, the designer adds metadata to the designated image, see p. 
10 of the Examiner's Answer. 

As discussed above, Madrane's tags are associated with the object not the region 
(e.g., the tags remain with the object, wherever it subsequently moves). Hence, in 
Madrane, the center of the screen is not permanently associated with metadata tags 
(which are added to the object when the object moves to the center of the screen), but 
instead it the object that is associated with a metadata tag when it moves to the center of 
the display. 

3. Third Reply Argument 

With respect to the issue of whether claims 1 and 15 are anticipated under Section 
102(b) by Madrane (US 6,573,907), and more specifically, with respect to the issue of 
whether Madrane teaches selectively moving a representation of a selected media object 
to region of the display, the Examiner stated that the disclosed feature in Madrane of 
designating different viewing positions to view a root image comprising various frames is 
the same as "applying a function to the root images to enable a user, through the applied 
functions, to specify a viewing position, with the result that the root images are displayed 
differently than if no function is applied. Thus, Madrane does teach selectively moving a 
representation of a selected media object to a region of the display", see p. 1 1 of the 
Examiner's Answer. 

The Examiner suggests that the images are displayed differently depending on 
what functions (metadata) have been applied to them. Thus the position and manner in 
which the object is displayed is a consequence of the tags applied. Conversely, in the 
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invention of claims 1 and 15, the tags applied are determined by the position to which the 
object has been moved by the user. In contrast, in Madrane, the user designates a position 
on the screen as a viewing position by merely clicking with the computer mouse. In other 
words, in Madrane, there is no moving of a selected media object to a specific region of 
the display. 

4. Fourth Reply Argument 

With respect to the issue of whether claims 1 and 1 5 are anticipated under Section 
102(b) by Madrane (US 6,573,907), and more specifically, with respect to the issue of 
whether the user can selectively move the objects appearing in the various video frames, 
the Examiner stated that it is the same to "selectively move objects" as to "watch a flow 
of a sequence of the video frames that include those objects", see p. 1 1 of the Examiner's 
Answer. 

Applicant submits that the passive watching of a sequence is not the same as 
intervening and controlling their movement oneself. In Madrane, the only control given 
to the user is to select a viewpoint, and then the user views the resulting display of the 
sequence. This is not the same as selecting an individual object and moving it around on 
the display. 

5. Fifth Reply Argument 

With respect to the issue of whether claims 1 and 15 are anticipated under Section 
102(b) by Madrane (US 6,573,907), and more specifically, with respect to the issue of 
whether the zooming of the root image in Madrane is the same as the claimed moving of 
the selected media object, the Examiner stated that in both the Madrane and the instant 
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application, "the illusion of a selected image moving across the screen is generated by 
'additional frames that are displayed, so that a more detailed version of the video 
sequence is shown' (e.g., in the case of a zoom function)", see p. 12 of the Examiner's 
Action. 

Applicant indeed acknowledges that Madrane teaches adding frames when 
performing a zoom function. However, this has no relevance to the invention of claims 1 
and 15. Madrane's "root image" remains in the same location in any given frame of the 
movie sequence, subject only to changes in viewpoint (zoom, pan etc.), which do 
not affect its position in its virtual world. The Examiner states that "in both the instant 
application and Madrane, the illusion of an image moving across the screen is generated 
by displaying additional frames so that a more detailed version of the video sequence is 
shown". Madrane certainly discloses this, but the present application does not, and 
certainly does not require it. Instead, claims 1 and 15 require a representation of an object 
(e.g. a thumbnail or icon representing the media object to be moved, under the control of 
a user, from part of a display to another). This is not a video (movie) sequence. The icon 
may represent any media object: audio, video, still photograph, text, numerical data etc. 
(see page 1 lines 8-10 of the instant specification). 

Claim 1 (similarly for claim 1 5) requires "a representation of a selected media 
object to be selectively moved by a user into a region of the display representing a 
selected set of metadata tags". In Madrane, the root image itself does not move, just a 
portion of the root image may be enlarged by inserting additional frames in the sequence. 

6. Sixth Reply Argument 
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With respect to the issue of whether claims 1 and 15 are anticipated under Section 
102(b) by Madrane (US 6,573,907), and more specifically, with respect to the issue of 
whether in Madrane it is the designer (and not the user) who sets values for the metadata 
tags, and whether in Madrane an action on the objects (e.g., two people walking toward 
each other) is caused because of metadata tags associated with the objects previously, the 
Examiner stated that in Madrane the designer associates metadata tags with objects and 
then a user is able to select the objects. "Thus... a user is able to perform this action 
'because of the metadata tags associated with those objects'". Moreover, the Examiner 
stated that it is readily apparent that he meant the term "user" to mean "designer" in his 
arguments in the Office Action of September 4, 2009, see p. 13 of the Examiner's 
Answer. 

It is apparent that Madrane describes two different processes: first the initial 
creation of the movie sequence, and application of the tags by the designer, and secondly 
the selection of the viewpoint, zoom scale etc., from which to view the resulting scene by 
the user. The Examiner's interpretation relies on confusion between these separate roles. 
In the invention of claims 1 and 15, the media objects have no metadata tags until the 
end-user applies them, by moving their representations to the location associated with a 
particular tag. In other words, unlike Madrane, there will be no addition of metadata tags 
to an object unless the object is moved by the end-user to the display area having this 
metadata tag associated with it. In Madrane, regardless of what the end-user does, a 
metadata tag is added to an object (by the designer). 

7. Seventh Reply Argument 
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With respect to the issue of whether claims 1 and 15 are anticipated under Section 
102(b) by Madrane (US 6,573,907), and more specifically, with respect to the issue of 
whether a selected set of metadata tags is added to the object depending on where the 
object is, the Examiner stated that in Madrane a designer, before release to the end-user, 
can tag objects in a video frame, tag a selective video frame, and tag a root image, see p. 
14 of the Examiner's Answer. 

In Madrane, the tagging is done in association with an action to be performed (i.e. 
that may be performed by the end-user), such as displaying a particular object, zooming 
in on it, etc. However the passage referred to be the Examiner (e.g., col. 16, lines 25-33) 
does not teach that the designer moves objects around to assign a particular action to a 
user selection. The designer tags part of each frame to identify where a particular object 
appears in that frame, so that it may subsequently be tracked as the viewer (end-user) 
watches a sequence of frames. The designer does not respond to the user selection - 
the selection made by the user simply identifies which object (which has previously been 
tagged by the designer) is to be tracked. 

8. Eighth Reply Argument 

With respect to the issue of whether claims 8 and 21 are anticipated under Section 
102(b) by Madrane (US 6,573,907), and more specifically, with respect to the issue of 
whether Madrane teaches applying a metadata tag set associated with a second display 
area to a copy of a media object moved there, the Examiner stated, as he did in point 7 
above, that a designer, before release to the end-user, can tag objects in a video frame, tag 
a selective video frame, and tag a root image, see p. 15 of the Examiner's Answer. 
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Madrane is completely silent as to applying metadata tags to copies of objects. At 
most, Madrane teaches controlling the display of a sequence of video frames depending 
on a designation by the user. 

9. Ninth Reply Argument 

With respect to the issue of whether claims 13 and 27 are anticipated under 
Section 102(b) by Madrane (US 6,573,907), and more specifically, with respect to the 
issue of whether Madrane teaches "making the size of the display area allocated to each 
set of metadata tags proportional to the number of media objects portrayed therein", the 
Examiner stated that Madrane teaches a user-designated viewing position to enable 
allocation in the display of each set of objects carrying metadata tags proportional to the 
number of media objects portrayed therein, see p. 15 of the Examiner's Answer. 

Claims 13 and 27 require the various display areas ("bins") on the screen to adapt 
in size to the relative number of objects in each of those bins. The Examiner equates this 
to a zoom function in which, by zooming out of a scene, more objects become visible. In 
the invention of claims 13 and 27, each display area, which corresponds to a particular 
tag or combination of tags, is given an area proportional to the number of objects to 
which that tag (or combination) has been applied. This allows all the icons, thumbnails, 
or other representations, to be depicted as having the same size, by changing the size of 
each bin to fit its own population. In contrast, in Madrane, even in a zoom out operation, 
the size of the display area, e.g., the square frame shown in Figs. 1, 4-7, does not change 
in size . It is the viewing angle that changes that causes more objects within the square 
frame to be displayed. 
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For at least the reasons set forth above and discussed in detail in the previously 
filed Appeal Brief, it is respectfully requested that the rejections on appeal be reversed. 

Respectfully submitted, 
NIXON & VANDERHYE P.C. 

By: /Leonidas Boutsikaris/ 

Leonidas Boutsikaris 
Reg. No. 61,377 



LB:tlm 

901 North Glebe Road, 1 1th Floor 
Arlington, VA 22203-1808 
Telephone: (703) 816-4000 
Facsimile: (703)816-4100 
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